Adaptive tuning of numerical weather prediction models: Randomized GCV in three and four dimensional data assimilation

In variational data assimilation, optimal ingestion of the observational data and optimal use of prior physical and statistical information involve the choice of numerous weighting, smoothing and tuning parameters which control the ltering and merging of divers sources of information. Generally these weights must be obtained from a partial and imperfect understanding of various sources of errors, and are frequently chosen by a combination of historical information, physical reasoning and trial and error. Generalized Cross Validation (GCV) has long been one of the methods of choice for choosing certain tuning, smoothing, regularization parameters in ill-posed inverse problems, smoothing and ltering problems. In theory, it is well suited for the adaptive choice of certain parameters that occur in variational objective analysis, and data assimilation problems that are mathematically equivalent to variational problems. The main drawback of the use of GCV in data assimilation problems was that matrix decompositions were apparently needed to compute the GCV estimates. This limited the application of GCV to data sets of the order of less than about 1000. Recently the randomized trace technique for computing the GCV estimates has been developed, and this makes the use of GCV feasible in essentially any variational problem which has an operating algorithm to produce estimates, given data. In this paper we demonstrate that the answers given by the randomized trace estimate are indistinguishable in a practical sense from those computed more exactly by traditional methods. Then we carry out an experiment to choose one of the main smoothing parameters ( ) in the context of a variational objective analysis problem which is approximately solved by k iterations of a conjugate gradient algorithm. We show how the randomized trace technique can be used to obtain good values of both and k in this context. Finally, we describe how the method can be applied in operational-sized three and four dimensional variational data assimilation schemes, as well as in conjuction with a Kalman lter. i